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A METHOD AND SYSTEM FOR 
ADAPTIVE TARGET DETECTION 

Technical Field 

[0001] The present invention relates generally to image processing. It 
particularly relates to an image processing target detection system and method 
that uses adaptive spatial filtering and time-differencing processes to detect and 
track targets within various background environments. 

Background of the Invention 
[0002] Passive IR (Infrared) sensors are widely used to detect the energy 
emitted from targets, backgrounds, incoming threats, and the atmosphere for a 
plurality of applications including military surveillance, missile target and detection 
systems, crop and forest management, weather forecasting, and other 
applications. The measures of performance for passive IR sensors include signal- 
to-noise ratio (S/N), radiation contrast, noise-equivalent temperature difference 
(NEDT), minimum resolvable temperature difference, and other parameters. 
These sensors may be designed to enhance one or more of these parameters for 
optimum performance during a particular application. 

[0003] Particularly, one type of passive IR sensor, the IRST sensor (Infrared 
search and track), locates and tracks objects by capturing the energy emitted 
within the field of view (FOV) or field of regard (FOR) of the sensor. However, 
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IRST sensors are commonly designed to operate with a small noise-equivalent 
temperature difference (NEDT) to detect small target-to-background contrast 
temperatures, and therefore heavy background clutter may strongly hinder 
accurate target detection and tracking and lead to a higher probability of false 
alarm (Pfa). Importantly for threat detection applications, it is useful for the IRST 
sensor to detect, declare, and track airborne targets at a long distance (usually 
larger than 50 km) before the threat can see the intended target, and therefore the 
IRST sensor performance may be enhanced using a large instantaneous field of 
view (e.g., 360-degree hemisphere in azimuth and 50 to 90 degrees in elevation). 
However, the large number of scene pixels produced by an IRST sensor may 
require computer-controlled image data processing to separate the large number 
of false targets from the true targets. As shown in FIG. 1A, a common target 
detection and tracking scenario for military applications may be a fighter jet 109 
attempting to detect and track incoming fighter jets 122 and/or incoming missiles 
(bombs) 124 that may be enemy-controlled. 

[0004] Commonly, the IRST sensor uses two image data processing 
techniques for target (threat) detection and tracking which include SpatiallRST 
and ChangelRST. FIG. 1B illustrates an exemplary SpatiallRST image 
processing system 100 found in the prior art. During operation, an image 102 
input from an IR sensor (not shown) is initially spatially convolved by a matched 
filter 104 to generate a spatially filtered image output. The matched filter 104 may 
be generally designed using a well-known system point spread function (PSF) 
since at a long distance an incoming airborne target may be considered as a point 
radiant source. A point spread function maps the intensity distribution for the 
received signal at the sensor generated from the point source of light (airborne 
target at a long distance). The spatially filtered output may be divided by a local 
background estimation (provided by an estimator 106) using a divider 108 which 
provides an output image to a CFAR (constant false alarm rate) detector 110. 
Use of a CFAR detector allows for setting of one or more detection threshold 
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levels to provide a maximum (tolerable) false alarm rate. The detector 110 
provides an output signal 112 indicating detection. 

[0005] However, SpatiallRST may produce a lot of false alarms when the 
background clutter contains high spatial frequency components. Also, when the 
background contains both low and heavy clutter sub-regions, traditional 
SpatiallRST may produce increased false alarms for the heavy clutter sub-regions 
which also reduces the probability of detection for the low clutter sub-regions. 
[0006] For light or medium background clutter, generally the SpatiallRST 
system works well to detect and track targets, but performance suffers with heavy 
to extremely heavy background clutter (e.g., urban and earth object clutter) 
leading to a high P fa . Under these conditions, commonly a ChangelRST image 
processing system may be used which employs a temporal time-differencing 
image processing technique which is useful for moving (e.g., airborne) targets. 
FIG. 2 illustrates an exemplary ChangelRST image processing system 200 found 
in the prior art. During operation, a reference image (current image frame) 202 
and a previous image (the search image) 204 are filtered using a high-pass filter 
206 and pixel-wisely registered using a registering device 208 at a particular re- 
visit time (RT). Pixel registration is a well-known technique to align the received 
images for the same scene. Commonly, a base image is used as a comparison 
reference for at least one other (input) image, and the registration process brings 
the input image into alignment with the base image by applying a spatial 
transformation to the input image. Using a subtractor 210, the registered search 
image may be subtracted from the reference image to suppress background 
clutter, and the output difference image may be fed to a CFAR (constant false 
alarm rate) detector 212 to generate a detection output signal 214. 
[0007] Alternatively, another ChangelRST image processing system 300 found 
in the prior art may be used as shown in FIG. 3. During operation of the 
alternative arrangement 300, an original large image 302 is under-sampled using 
a sampler 304 into a smaller matrix containing match point elements. These 
match point elements are registered using registering device 208, and the 
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registration locations are interpolated back to the original space-domain. After 
interpolation, operation continues similar to FIG. 2 with the subtractor 210 to 
generate a difference signal input to CFAR detector 212. This alternative 
ChangelRST arrangement 300 uses a multi-resolution approach to reduce the 
throughput (computing load) requirement for the image registration. However, the 
registration accuracy is decreased. 

[0008] Therefore, due to the disadvantages of current image processing 
techniques for IRST sensors, there is a need to provide an adaptive image 
processing system that provides high probability of detection in various 
background environments (light, medium, or heavy clutter) while maintaining low 
probability of false alarm. 

Summary of the Invention 
[0009] The method and system of the present invention overcome the 
previously mentioned problems by providing a target detection and tracking 
system capable of providing adaptive image processing for an IRST sensor 
system. The adaptive image processing includes an adaptive spatial filtering 
technique that uses high-pass filtering and adaptive thresholding to reduce the 
false alarm rate in the presence of background clutter containing high spatial 
frequency components. The adaptive spatial filtering technique may be combined 
with a spot time-differencing technique that performs time-differencing processing 
only for areas of detection in high clutter sub-regions based on the adaptive 
spatial filtering results which maintains a low false alarm rate for light clutter sub- 
regions. 

Brief Description of the Drawings 
[00010] Fig. 1A is a block diagram of an exemplary target detection and 
tracking scenario for military applications found in the prior art; 
[00011] Fig. 1B is a block diagram of an exemplary target detection image 
processing system using spatial filtering found in the prior art; 



-4- 



Attorney Docket No. 4358-01 17P 

[00012] Fig. 2 is a block diagram of an exemplary target detection image 
processing system using time-differencing found in the prior art; 
[00013] Fig. 3 is a block diagram of an exemplary, alternative target 
detection image processing system using time-differencing found in the prior art; 
[00014] Fig. 4 is a flow process diagram of an exemplary adaptive IRST 
image processing system in accordance with an embodiment of the present 
invention. 

[00015] Fig. 5 is a block diagram of an exemplary adaptive IRST image 
processing system using adaptive spatial filtering in accordance with an 
embodiment of the present invention. 

[00016] Fig. 6 is a block diagram of an exemplary adaptive IRST image 
processing system using spot time-differencing in accordance with an 
embodiment of the present invention. 

[00017] Fig. 7 shows an illustration of exemplary background clutter images 

in accordance with an embodiment of the present invention. 

[00018] Fig. 8 shows an exemplary point spread function of the optical IRST 

system in accordance with an embodiment of the present invention. 

[00019] Fig. 9 is an exemplary illustration of target locations in an IRST 

system in accordance with an embodiment of the present invention. 

[00020] Figs. 10-14 show graphs with exemplary IRST performance sensor 

sensitivity curves for adaptive spatial filtering and spot time-differencing in 

accordance with an embodiment of the present invention. 

Detailed Description 
[00021] Fig. 4 is a flow process diagram of an exemplary adaptive IRST 
image processing system in accordance with an embodiment of the present 
invention. Advantageously, a controller may be used to control the flow process 
steps of the IRST imaging system. At step 402, a reference (current) image frame 
and a search (previous) image frame may be input, from an IRST sensor, into the 
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system using a receiver and undergo image pre-processing including noise 
filtering and other pre-processing. 

[00022] In an exemplary embodiment, the reference image may be received 
at a time (t) and the previous image may be received at a previous time (t - n). At 
step 404, the reference image is input to an adaptive spatial filtering path (further 
described below in reference to FIG. 5) for detection of an object within the sensor 
field of view (e.g., impending threat such as launched missiles, etc.). At step 406, 
a decision block is reached where it is determined whether the background clutter 
in the field of view qualifies as high (heavy) clutter in accordance with a 
predetermined threshold. If yes, then processing continues at step 408 where 
spot time-differencing processing (spot ChangelRST) is performed on the 
reference and search images to reduce the stationary detections due to clutters 
(such as building and rocks, etc.) and to pass moving detections (such as airborne 
targets). 

[00023] Following at step 410, the confirmation detection from the spot time- 
difference step (step 408) may be combined with the detections with low clutter 
("no" decision at step 406) from the spatial filtering step (step 404) to produce a 
detection summation output. At step 412, extended image processing including 
classification, identification, and tracking may occur using the summation 
detection result and the reference image as inputs to initiate and maintain tracking 
of the detected object. 

[00024] Fig. 5 is a block diagram of exemplary adaptive IRST image 
processing system 500 using adaptive spatial filtering in accordance with an 
embodiment of the present invention. Advantageously, adaptive IRST image 
processing system 500 may be used for the detection/search scenario illustrated 
in FIG. 1A to replace the prior art systems 100, 200, 300 shown in FIGs. 1B, 2, 3. 
A controller 509 may be used to control the operation of the system 500. 
[00025] As shown in Fig. 5, a reference (current) image frame 502 may be 
input from an IRST sensor field of view (not shown) to a spatial, matching filter 
504 using a receiver 507. Advantageously, spatial filter 504 may perform high- 
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pass filtering using a smaller template (incoming pixel frame size for the filter) 
which enables faster detection by requiring less processing than for larger size 
templates. The filter 504 operates to use a previously detected object (e.g., tank) 
as the center for the succeeding pixel frame of a limited size (smaller template) 
which accelerates accurate correlation and detection. Also, spatial filter 504 may 
subtract the original image from a local mean to function as an anti-mean high- 
pass filter. 

[00026] Additionally, a background estimator 506 may estimate the noise of 
the background clutter of the IRST sensor field of view using the same anti-mean 
filter 504 or using a different high-pass filter (e.g., the filter of a point spread 
function), and divide (using divider 508) the filtered image data input by the 
background noise estimation to produce an output image signal input to a CFAR 
(constant false alarm rate) detector 510. Use of a CFAR detector allows for 
setting of one or more detection threshold levels to provide a maximum (tolerable) 
false alarm rate for the system 500. Advantageously, anti-mean filter 504 with a 
smaller template may reduce the false alarm rate when the background clutter of 
the sensor field of view contains high frequency components. 
[00027] Also, the reference image data 502 may be input to a local/regional 
sigma (standard noise deviation) estimator 512 to help estimate the standard 
deviation for noise within the background clutter for the field of view. The 
estimator 512 divides the image data 502 into a plurality of different spatial sub- 
regions and determines (measures) the SNR and standard noise deviation for 
each sub-region including a local region. Following the estimator 512, threshold 
device 514 may set the SNR threshold levels for each sub-region based on the 
measurements of the estimator 512. Following, the CFAR detector 510 may 
receive the noise estimation and SNR threshold levels, along with the 
filtered/divided image data signal output, to determine whether an object is 
detected (e.g., predetermined threat target) within the sensor field of view and 
produces a detection output signal 516. 
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[00028] Following generation of the detection output signal 516, image 
processing may continue using the spot time-differencing system 600 of FIG. 6. 
Fig. 6 is a block diagram of an exemplary adaptive IRST image processing system 
600 using spot time-differencing in accordance with an embodiment of the present 
invention. As shown in FIG. 6, the reference image 502 and a search (previous) 
image 601 input to the spatial filter 504 of system 500 may be also input to a high- 
pass filter/background estimator device 602 for filtering and estimating of the 
noise level for the background clutter across the plurality of sub-regions within the 
sensor field of view. The processing of system 600 continues if high clutter is 
determined (step 406 from FIG. 4) for the particular sub-regions since 
advantageously spot time-differencing will be applied for detection confirmation in 
only high background clutter sub-regions. Following, the filtered reference and 
search image data 502, 601 are input to a registrator 604 for registering of pixel 
data for the input image data 502, 601 for proper alignment of images from the 
same scene (field of view). The registrator 604 compares the input image data 
502, 601 with base image data to determine whether spatial transformation of the 
input image data is necessary for proper alignment with the base image data. 
Thereafter, a differencer 606 may subtract the search image 601 from the 
reference image 502 to suppress background clutter, and the output difference 
image 603 fed to a CFAR detector 608 to generate a detection output signal 609 
indicating whether an object (e.g., predetermined threat target) is detected in 
sensor field of view. 

[00029] As shown at step 412 of FIG. 4, extended image processing 
including classification, identification, and tracking may occur using the spatial 
filtering processing output, time-difference detection output, and original reference 
image data as inputs to initiate and maintain tracking of the detected object. 
[00030] Fig. 7 shows an illustration of exemplary background clutter images 
in accordance with an embodiment of the present invention that may require 
system 500, 600 for accurate target detection. The images of FIGs. 7A, 7B show 
background clutter of a mountain view where FIG. 7B shows an image (search 
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image) 704 collected one frame before the image (reference image) 702 in FIG. 
7 A where the revisit-time between the two images may be approximately 0.33 
seconds. FIG. 7C shows the difference image 706 obtained by subtracting the 
search image 704 of FIG. 7B from the reference image 702 of FIG. 7A. For this 
example, the difference image 706 may produce reduced background clutter 
(reduced standard noise deviation) to provide a higher probability of detection of 
the airborne target. 

[00031] Fig. 8 shows an exemplary point spread function (PSF) 800 of the 
optical IRST system in accordance with an embodiment of the present invention. 
The PSF is created by considering the incoming airborne target (at a far distance 
away) as a point radiant (light) source, and mapping the intensity distribution for 
the received signal at the sensor. For an exemplary embodiment, to detect a SMF 
(small military fighter) target at a far distance away (e.g., greater than 50 km), the 
target may be considered a point source (sub-pixel detection). FIG. 8 shows the 
PSF 800 representing the energy distribution of the IRST sensor (e.g., IR focal 
plane array - IR FPA) after a point source (e.g., small military fighter) passes 
through the optical lens of the system. PSF 800 represents the degree of 
degradation as the light passes through the optical lens of the system since the 
system optics are not perfect. For example, a PSF with a contrast of 16 counts 
(image counts-to-the-irradiance factor) may be related to a SMF target signature 
at a distance of 69 km away from the IRST sensor. Fig. 9 is an exemplary 
illustration of 25 SMF target locations 900 with contrasts of 16 counts randomly 
(with a uniform distribution) inserted into the reference image 702 of FIG. 7a. 
[00032] Figs. 10-14 show graphs with exemplary IRST performance sensor 
sensitivity curves for adaptive spatial filtering and spot time-differencing in 
accordance with an embodiment of the present invention. Relying on 
predetermined measurements and analysis (e.g., testing and/or computer 
simulation of sensor operation), comparative received operating characteristics 
(ROC) between each sensor in the IRST system 500, 600 (for a multi-sensor 
system) may be calculated during a predetermined tracking period. The ROC 
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performance (curve) for each one of the plurality of sensors may be generated 
using likelihood (probability) functions to represent sensor information during 
target tracking such as target detections, no detections, measured SNRs, and 
other sensor information obtained from sensor measurements, observations, or 
other sensor data outputs. 

[00033] FIGs. 10a, 10b show the receiving operating characteristics (ROC) 
curves for system 500 using three different anti-mean filters. For example, the 
three different filters may be 1 X 3 row, 3X3 square, and 3 X 1 column. As 
shown in these figures, the row filter may generate the best performance with a P d 
(probability of detection) approximately 95% and a Pf a (probability of false alarm) 
approximately 4E-4. Also, FIGs. 11a, 11b show the detection performance for the 
row filter. As shown in these figures, the row filter generates high P d and low Pf a . 
[00034] FIGs. 12, 13 shows the ROC curves for background clutter of 
medium urban earth (FIG. 12) and heavy rural earth (FIG. 13) for an exemplary 
embodiment of systems 500, 600. Using a row filter and an SMF (small military 
fighter) at a range of 69 km, system 500, 600 reduces Pf a to approximately 5E-4 
and increases Pd to approximately 80% as shown in FIG. 12. Also, the false- 
alarm number is approximately 240/frame for a 724 X 724 FPA (focal point array). 
As shown in FIG. 13, systems 500, 600 reduce Pf a to 5.7E-3 and increase P d to 
approximately 80%, and the false-alarm number is approximately 3010/frame for a 
724 X 724 FPA. By introducing a spot time-differencing process (implemented 
only if a particular clutter threshold is reached at step 406), only several hundred 
template registrations/correlations need to be performed for medium clutter 
background, and only approximately 3,000 for heavy clutter background. 
[00035] FIG. 14 shows the ROC curves 1400 for a medium urban earth 
background clutter for exemplary embodiments of systems 500, 600. As shown in 
FIG. 14, a high P d of 80% is maintained, and the P fa is reduced to 3.5E-3 for the 
adaptive spatial filtering (SP-IRST) of system 500, and further reduced to 9.0E-4 
for the spot time-differencing (TD-IRST) that is applied by system 600. 
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[00036] A plurality of advantages may be provided by the invention 
described herein. Using an adaptive spatial filtering and spot time-differencing 
digital image processing algorithm, faster throughput may be realized for heavy 
background clutter in the reference image data as the number of time-differencing 
correlations are greatly reduced by limiting the time-differencing application to a 
heavy clutter environment. Also, local and regional noise standard deviation 
estimation allows adaptive generation of sub-region SNR thresholds to reduce 
false alarms and increase probability of detection in the sub-regions of the 
reference image data. Further advantages may also be realized including higher 
image registration accuracy and other advantages. 

[00037] Although the invention is primarily described herein using particular 
embodiments, it will be appreciated by those skilled in the art that modifications 
and changes may be made without departing from the spirit and scope of the 
present invention. As such, the method disclosed herein is not limited to what has 
been particularly shown and described herein, but rather the scope of the present 
invention is defined only by the appended claims. 
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